Estimation Error in Image Quality Measurements

The development and adoption of standard image quality measurement and analysis methods have helped both the evaluation of competing imaging products and in technologies. Inherent in the interpretation of results from any particular evaluation, however, are the variation of the method itself, the sampling of test images, equipment, and test conditions. Here we take a statistical approach to measurement variation, and interpret the objective as being the estimation of particular system or image properties, based on data, collected as part of standard testing. Measurement variation was investigated for two signal-transfer methods commonly used for digital camera and scanner evaluation: the ISO 12233 slanted-edge spatial frequency response and the dead-leaves method for texture-MTF evaluation being developed by the Camera Phone Image Quality (CPIQ) Initiative. In each case, the variation due to the selection of analysis regions was computed by repeated analysis. The slanted-edge methods indicated a relative error in the range of 1-3% depending on the nature of the region selection. For the dead-leaves method, the amplitude spectrum (square root of the noise-power spectrum) showed a relative error of approximately 4-6%, however, this can be reduced by applying spectral estimation methods commonly used in image noise analysis.